Introducing Low-Stakes Just-in-Time Assessments to a Flipped Software Engineering Course

Objective: We present a Teaching-as-Research project that implements a new intervention in a flipped software engineering course over two semesters. The short-term objective of the intervention was to improve students’ preparedness for live sessions. The long-term objective was to improve their knowledge retention evaluated in time-separated high-stakes assessments. Intervention: The intervention involved adding weekly low-stakes just-in-time assessments to course modules to motivate students to review assigned instructional materials in a timely manner. The assessments consisted of, per course module, two preparatory quizzes embedded within off-class instructional materials and a non-embedded in-class quiz. Method: Embedded assessments were deployed to two subgroups of students in an alternating manner. In-class assessments were deployed to all students. The impact of embedded assessments on in-class assessments and on final exam performance was measured. Results: Embedded assessments improved students’ preparedness for live sessions. The effect was statistically significant, but variable. Embedded assessments did not impact long-term knowledge retention assessed on final exam. We have decided to keep the intervention and deploy it to all students in the future

Topic selection in industry experiments

This paper shares our experience with initial negotiation and topic elicitation process for conducting industry experiments in six software development organizations in Finland. The process involved interaction with company representatives in the form of both multiple group discussions and separate face-to-face meetings. Fitness criteria developed by researchers were applied to the list of generated topics to decide on a common topic. The challenges we faced include diversity of proposed topics, communication gaps, skepticism about research methods, initial disconnect between research and industry needs, and lack of prior work relationship. Lessons learned include having enough time to establish trust with partners, importance of leveraging the benefits of training and skill development that are inherent in the experimental approach, uniquely positioning the experimental approach within the landscape of other validation approaches more familiar to industrial partners, and introducing the fitness criteria early in the process

Architecture-Driven Verification of Concurrent Systems

This paper proposes a method to construct a set of proof obligations from the architectural specification of a concurrent system. The architectural specifications used express correctness requirements of a concurrent system at a high level without any reference to component functionality. Then the proof obligations derived from such specifications are discharged as model checking tasks in a suitable behavioral model where components are assigned their respective functionalities. An experimental extension to the SPIN tool is used as the model checker. The block diagram notation used to specify architectures allows interchangeable components with equivalent intended functionalities to be encapsulated within a representative module. A proof obligation of such a system is discharged as an equivalence checking task in the behavioral model chosen. It is shown how infeasible proof obligations can be decomposed by decomposing the architectural specification. Obligation decomposition relies on assume-guarantee conditions

Valuation of Learning Options in Software Development Under Private and Market Risk

Commercial software development is an inherently uncertain activity. Private risk is high, schedule and cost overruns are common, and market success is elusive. Such circumstances call for a disciplined project evaluation approach. This paper addresses the of use of market and earned value management data in assessing the economic value of commercial software development projects that are simultaneously subject to schedule, development cost, and market risk. The assessment is based on real options analysis, a financial valuation technique that can tackle dynamic investment decisions under uncertainty. The paper demonstrates the application of real options analysis to a development scenario that consists of two consecutive stages: a mandatory prototyping stage and an optional full-development stage. The full-development stage is undertaken only if the prototype is successful and the market outlook is sufficiently positive at the end of the prototyping stage, thus giving the full-development stage the flavor of an option. The project's staged design increases its value. Real options analysis captures the extra value due to optionality.Le d\ue9veloppement de logiciels commerciaux est une activit\ue9 fondamentalement incertaine. Les risques sont \ue9lev\ue9s pour le secteur priv\ue9, le d\ue9passement des \ue9ch\ue9ances et des co\ufbts est fr\ue9quent, et la r\ue9ussite commerciale est rarement assur\ue9e. Il faut donc recourir \ue0 une m\ue9thode rigoureuse pour \ue9valuer le projet. Il est question dans cet article de l'utilisation des donn\ue9es sur la gestion des march\ue9s et de la valeur ajout\ue9e pour \ue9valuer la valeur \ue9conomique des projets de d\ue9veloppement de logiciels commerciaux qui comportent des risques quant aux d\ue9lais, aux frais de d\ue9veloppement et \ue0 la mise en march\ue9. L'\ue9valuation se fonde sur l'analyse des options r\ue9elles, m\ue9thode d'\ue9valuation financi\ue8re qui aide \ue0 prendre des d\ue9cisions d'investissement dynamique dans un contexte d'incertitude. L'auteur applique l'analyse des options r\ue9elles \ue0 un projet de d\ue9veloppement virtuel en deux \ue9tapes successives : une phase indispensable de prototypage et une phase facultative de d\ue9veloppement complet. On amorce la seconde phase que si la premi\ue8re est couronn\ue9e de succ\ue8s et si les d\ue9bouch\ue9s commerciaux semblent suffisamment int\ue9ressants, ce qui laisse le choix de passer ou non au d\ue9veloppement complet. La conception par \ue9tapes d'un projet en accro\ueet la valeur, et l'analyse des options r\ue9elles permet de d\ue9finir la valeur ajout\ue9e qui leur est attribuable.NRC publication: Ye